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SUMMARY  PAGE 


THE  PROBLEM 

Naval  flight  officer  candidate  attrition  in  undergraduate  training  has 
increased  from  a  low  of  20  percent  in  the  early  1960s  to  more  than  40  percent 
within  the  last  five  years.  One  means  of  reducing  attrition  is  to  increase  the 
efficiency  of  the  Naval  and  Marine  Aviation  Selection  Test  Battery  used  to  select 
candidates  for  the  various  naval  flight  officer  (NFO)  aviation  training  pro¬ 
grams.  The  scoring  key  tor  the  Biographical  Inventory  (BI)  developed  for  a 
pilot  population  is  currently  being  used  to  select  both  aviators  and  naval 


flight  officers  into  naval  aviation  training  programs.  Previous  experience 
with  biographical  data  suggested  that  a  scoring  key  developed  specifically  for 
the  NFO  subject  population  should  increase  the  efficiency  of  the  Biographical 
Inventory  selection  test.  The  present  report  represents  an  evaluation  of  such 
a  scoring  key . 

FINDINGS 

Results  indicated  that  a  Biographical  Inventory,  specifically  keyed  for 
a  naval  flight  officer  population,  with  aerospace  knowledge  questions  excluded 
was  a  better  predictor  of  naval  flight  officer  candidate  attrition  than  the  Bio¬ 
graphical  Inventory  keyed  for  a  pilot  population.  This  predictive  superiority 
was  lessened  when  aerospace  knowledge  questions  were  omitted  from  the  con¬ 
ventional  BI  score . 

RECOMMENDATIONS 

It  is  recommended  that  the  newly  developed  NFO  BI  key ,  with  aerospace 
knowledge  questions  excluded,  be  considered  for  use  in  the  Naval  and  Marine 
Aviation  Selection  Test  Battery.  If  the  newly  keyed  BI  is  not  adopted  for  NFO 
selection,  then  it  is  suggejted  that  the  scoiing  of  aerospace  knowledge 
questions  be  omitted  from  the  BI  presently  used  for  this  population  in  order 
to  increase  the  effectiveness  of  naval  flight  officer  selection  . 
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INTRODUCTION 


In  the  Naval  Aviation  Training  Program ,  attrition  is  considered  a  serious 
problem,  since  it  represents  both  a  financial  and  a  manpower  loss.  Attrition 
refers  to  those  students  who  are  removed  from  training  because  of  academic  or 
inflight  performance  difficulities ,  and  to  those  who  voluntarily  withdraw  from 
training.  Within  the  training  program  ,  there  are  two  basic  types  of  students: 
student  naval  flight  officers  (NFOs)  and  student  naval  aviators  (SNAs)  .  Of 
these  two  student  types,  the  attrition  rate  from  undergraduate  training  is  much 
higher  for  NFOs  than  for  SNAs.  Griffin  and  Mosko  (2)  reported  that  the  aver¬ 
age  attrition  rate  for  pilots  was  25  percent  from  1962  through  1976,  while  the 
NFO  attrition  rate  had  increased  from  a  low  of  20  percent  in  the  early  1960s  to 
more  than  40  percent  within  the  last  five  years. 

One  means  of  reducing  attrition  is  to  increase  the  efficiency  of  personnel 
selection  for  NFO  training.  Naturally,  early  identification  of  potential  attrites 
provides  the  best  savings.  The  Naval  and  Marine  Aviation  Selection  Test  Bat¬ 
tery  is  used  as  an  initial  screening  device  to  select  from  potential  candidates 
those  individuals  most  likely  to  succeed  in  training.  The  battery  consists  of 
four  separate  paper-and-pencil  tests:  Academic  Qualification  Test  (AQT)  , 
Mechanical  Comprehension  Test  (MCT)  ,  Spatial  Apperception  Test  (SAT)  ,  and 
the  Biographical  Inventory  (BI)  (1)  . 

The  Biographical  Inventory  is  the  only  instrument  in  the  test  battery 
with  noncognitive  type  questions.  Currently  the  BI  scoring  key  used  to  select 
both  pilots  and  NFOs  was  developed  on  the  pilot  population.  The  current 
(1971-1974)  validity  of  the  BI  key  for  the  two  student  types  is  shown  in  Table 
I.  The  zero-order  correlation  of  the  BI  was  related  to  the  training  Pass/Attrite 

Table  I 

Biographical  Inventory  Validity  Coefficients  for  a 
Pilot  and  NFO  Population 
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Criterion/ 

Sample 

Predictor 

Variables 

Zero 
Order  r 

Multiple 

R 

R2 

Increase 
in  R 

Pass/Attrite 

AQT 

.082 

.082 

.007 

.082 

Pilot 

MCT 

.117 

,127 

.016 

.045 

N  =  702 

SAT 

.109 

.153 

.023 

.026 

BI 

.212 

.245 

.060 

.092 

Pass/Attrite 

AQT 

.172 

.172 

.029 

.172 

NFO 

MCT 

.221 

.234 

055 

.062 

N  =  1039 

SAT 

.166 

.241 

.058 

.207 

BI 

.156 

.244 

.060 

.003 

criterion  for  both  groups .  The  BI  for  the  NFO  group  does  not  account  for 
additional  unique  variance  in  a  multiple  regression  equation  beyond  that  pro¬ 
vided  by  the  other  selection  tests.  However,  for  the  pilot  group  the  BI 
increased  the  explained  variance  by  27  percent.  As  a  result  of  these  findings 
a  new  scoring  key  was  developed  for  the  NFO  population  to  increase  the 
efficiency  of  the  BI  selection  test.  The  development  of  the  NFO  BI  key  and  its 
subsequent  evaluation  are  the  subjects  of  the  present  study. 


PROCEDURE 


SUBJECTS 


Records  for  all  NFO  candidates  who  entered  che  Naval  Aviation  Training 
Program  between  July,  1971,  and  January,  1974,  were  extracted  from  the 
Naval  Aviation  Medica.  Research  Laboratory  (NAMRL)  databank.  NFO  candi¬ 
dates  are  recent  college  gi^duates  who  enter  training  directly  from  civilian 
life  without  prior  military  training .  It  is  this  group  of  NFOs  who  have  the 
highest  attrition  rate  and  is  the  population  of  major  concern.  Of  the  1,039 
subjects  whose  records  were  available  for  analysis ,  509  completed  training  and 
530  attrited,  for  an  attrition  rate  of  51  percent. 

BIOGRAPHICAL  INVENTORY 

The  BI  is  an  untimed,  multiple-chcice  questionnaire  containing  160 
questions .  The  first  110  questions  pertain  to  biographical  information  and  the 
last  50  to  general  aerospace  knowledge.  There  are  two  forms  of  the  question¬ 
naire  with  a  different  set  of  aerospace  knowledge  questions  in  each  form  .  The 
biographical  questions  are  identical  in  the  two  tests  and  are  the  questions  of 
concern  for  development  of  the  NFO  BI  key. 

For  the  BI  scoring  key  the  biographical  alternatives  which  discriminate 
between  successfv.1!  and  unsuccessful  candidates  are  assigned  weights  of  +1  or 
-1.  depending  upon  the  relationship  to  the  dichotomous  Pass/Attrite  criterion. 
The  aerospace  knowledge  questions  are  scored  as  correct,  or  incorrect,  +1 
or  0.  The  sum  of  the  weights  of  the  biographical  portion  and  the  number  of 
correct  aerospace  questions  comprise  the  total  BI  score. 


NFO  BI  KEY  DEVELOPMENT  METHODS 


When  subject  records  were  initially  evaluated,  two  major  problems 
became  apparent.  First,  as  shown  in  Table  I,  the  BI  did  not  account  for  addi¬ 
tional  variance  in  the  Pass/Attrite  criterion  beyond  that  provided  by  the  other 
selection  tests.  The  inter -correlation  matrix  of  the  selection  tests  and  the  Pass/ 
Attrite  criterion  in  Table  n  indicates  that  the  BI  is  highly  correlated  with  the 
MCT.  An  additional  multiple  regression  analysis  which  forced  the  BI  into  the 
equation  before  the  MCT  was  performed  to  see  if  the  two  tests  were  accounting 
for  a  unique  portion  of  the  variance.  In  this  analysis  the  MCT  did  not  contri- 
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Table  n 


Selection  Test  Correlation  Matrix 


AOT 

MCT 

SAT 

BI 

CRITERION 

PASS/ATTRITE 

AQT 

1.000 

.472 

.349 

.296 

.172 

MCT 

1.000 

.468 

.519 

.221 

SAT 

1.000 

.327 

.166 

BI 

1.000 

.156 

bute  to  the  multiple  regression  equation .  These  results  demonstrate  that  the 
MCT  and  BI  are  explaining  the  same  portion  of  the  criterion  and,  for  NFOs, 
have  no  independent  predictive  power . 

The  second  difficulty  concerned  the  size  of  the  subject  population  in 
relation  to  the  number  of  test  item  alternatives .  To  conduct  a  meaningful  item 
analysis ,  an  integral  process  in  the  development  of  a  new  scoring  key,  the 
subject  population  should  be  considerably  larger  than  the  number  of  alter¬ 
natives  to  avoid  obtaining  high  validity  coefficients  based  on  chance  differ¬ 
ences  .  With  noncognitive  tests  such  as  the  biographical  portion  of  the  BI, 
each  question  alternative  must  be  assessed  individually  since  there  is  no 
"correct11  response.  The  BI  contains  440  biographical  test  answer  alterna¬ 
tives.  While  the  NFO  subject  population  (1039)  is  substantial,  it  does  not  meet 
the  generally  accepted  standards  regarding  sample  size. 

To  overcome  these  difficulties ,  two  different  methodologies  were 
developed:  Residual  Criterion  and  Item  Analysis  by  Sampling .  These  methods 
are  discussed  in  the  following  text. 

Residual  Criterion 

Item  selection  for  the  individual  tests  in  the  selection  test  battery  used 
the  dichotomous  Pass/Attrite  training  criterion.  As  a  result,  each  test  contains 
items  related  to  the  same  criterion .  If  item  selection  was  based  on  that  part 
of  the  criterion  which  other  tests  have  not  already  explained ,  a  test  could  be 
more  uniquely  predictive.  It  was  decided  to  develop  an  NFO  BI  key  based  on 
an  item  analysis  which  used  the  residual  criterion  variance ,  that  portion  of  the 
criterion  which  is  not  predictable  from  other  tests  in  the  battery .  A  key 
developed  by  this  method  should  theoretically  maximize  explained  variance, 
since  overlap  with  the  other  tests  would  have  been  largely  eliminated. 

To  develop  the  new  criterion,  a  multiple  regression  analysis  was  per¬ 
formed,  using  a  forward  selection  procedure.  The  selection  test  scores,  exclud- 


ing  the  BI,  were  forced  into  the  equation  in  the  following  order:  AQT,  MCT , 
and  SAT .  The  resultant  multiple  regression  equation  was  then  used  to  obtain 
the  predicted  criterion  score  for  each  individual .  The  predicted  score  was 
subtracted  from  the  individual's  actual  criterion  score  to  obtain  the  remaining 
or  residual  criterion.  In  this  case,  the  true  criterion  scoie  was  1  for  success¬ 
ful  completion  of  training  and  0  for  attrite.  A  constant  value  of  2  was  added  in 
order  to  obtain  positive  values . 

Y'  =  A  +  (BiJAQT  +  (B2)MCT  +  (B3)SAT 

Residual  criterion  =  Y  -  Y'  +  2 

where 


Y'  =  predicted  score 
Y  =  criterion  score 


A  =  constant 


Item  Analysis 


Sampling 


=  beta  weight  for  AQT 
B2  =  beta  weight  for  MCT 
B3  =  beta  weight  for  SAT 


A  viable  approach  for  assessing  alternatives  of  a  test  question  is  to  deter¬ 
mine  the  discriminability  (D)  index  for  an  alternative.  With  this  technique, 
a  Di  score  is  computed  as  the  difference  between  proportions  of  the  two  criterion 
groups  choosing  a  given  alternative .  Due  to  the  restricted  sample  size ,  it  was 
decided  to  use  sampling  procedures  to  obtain  multiple  D  index  values  and  then 
use  the  binomial  expansion  rule  to  define  the  distribution  of  the  discrimin¬ 
ability  indexes  (Dj) .  This  approach  was  used  to  stabilize  validity  coefficients 
for  alternatives  and  to  estimate  the  error  involved  with  each  validity . 


One  hundred  samples,  containing  100  subjects  each,  were  randomly 
selected  from  the  data  pool.  No  subject  was  selected  twice  within  a  sample. 
For  every  sample,  a  Di  was  calculated  for  each  alternative  as  well  as  a  phi 
coefficient  of  the  alternative  with  the  criterion  scores .  Dj  was  calculated  as 
follows: 


Di  =  P_l  -  Po 

X  Y 

where 


Pi  =  number  of  successful  students  who  selected  the  alternative. 

Po  =  number  of  successful  students  who  did  not  select  the 
alternative. 


X  =  total  number  of  people  who  selected  the  alternative. 

Y  =  total  number  of  people  who  did  not  select  the  alternative. 


As  can  be  se8n  by  the  equation,  is  the  difference  between  the  percentage  of 
successful  students  from  the  total  group  who  selected  the  item  and  the  percent¬ 
age  of  successful  students  who  did  not  select  the  item .  Dj  ranges  in  value 
between  +1.  For  some  samples,  certain  alternatives  were  not  chosen  by  any 
respondents  or  were  chosen  by  everyone.  Since  there  was  no  variability  in  the 
alternative  responses,  Di  was  set  to  zero  whenever  X  or  Y  equaled  zerc. 

Each  alternative  had_a  distribution  of  D^'s  taken  over  100  samples. 

The  mean  of  all  Di  values,  D,  and  the  corresponding  standard  deviation  for 
each  alternative  were  calculated.  D  reflects  the  magnitude  of  the  proportional 
differences  across  samples  between  responses  of  successful  and  unsuccessful 
students .  The  larger  the  absolute  value  of  D ,  the  greater  the  discrimination 
of  the  alternative.  For  developing  a  BI  key,  a  positive  D  suggests  positive 
keying  while  a  negative  D  suggests  a  negative  keying .  The  standard  deviation 
of  the  sample  values  of  Di  reflects  the  stability  of  the  discrimination .  The  larger 
the  standard  deviation,  the  less  agreement  across  samples. 

Since  nothing  was  known  about  the  Di  distribution,  e.g. ,  how  large  a 
standard  deviation  is  "too  large,"  it  was  decided  to  use  a  measure  of  consistency 
of  the  direction  of  Di  differences  found  between  samples.  In  this  case,  the  num¬ 
ber  of  samples  with  positive  Di  values  minus  those  with  negative  Di  values  was 
divided  by  the  number  of  sample.  The  resultant  values  were  converted  to  per¬ 
centages  and  scaled  to  a  range  of  from  0  to  100.  The  centerpoint,  50,  indicated 
no  discrimination,  100  perfect  positive  discrimination  (i.e. ,  the  signs  of  Di 
were  positive  in  all  samples) ,  and  0  indicated  perfect  negative  discrimination. 
The  Z  statistic  was  used  to  describe  the  distribution  of  the  scale  consistency 
score . 


In  addition  to  D ,  an  average  reliability  phi  coefficient  or  point  bi-serial 
across  samples  was  obtained  as  a  measure  of  validity  or  each  alternative.  For 
BI  keying,  an  ideal  item  alternative  would  have  a  hty  .i  (100)  or  low  (0)  scaled 
consistency  score  with  a  Z  probability  less  than  .10  and  an  average  phi  coef¬ 
ficient  which  indicates  validity  for  the  alternative . 

METHOD 


In  order  to  evaluate  the  use  of  a  residual  criterion,  two  new  BI  keys  were 
developed,  one  based  on  an  item  analysis  using  the  Pass/Attrite  dichotomous 
criterion  and  the  other  using  the  residual  criterion.  The  item  analysis  samp¬ 
ling  procedure  was  used  for  both.  Item  selection  was  based  upon  the  scaled 
consistency  index,  the  Z  score,  the  average  phi  coefficient  or  point  biserial, 
and  background  knowledge  concerning  the  usefulness  of  the  alternatives.  In 
addition,  the  number  of  times  D^  equaled  0  was  taken  into  consideration  since 
the  statistical  measures  based  on  only  a  few  discriminating  samples  would  be 
unstable  and  therefore  unreliable.  The  newly  developed  BI  keys  were  then 
applied  to  the  same  NFOC  population  to  determine  the  proficiency  of  the  newly 
developed  scoring  procedures . 
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Since  the  pilot  BI  key  includes  the  aerospace  knowledge  questions  as 
part  of  the  total  score ,  the  new  NFO  BI  keys  were  evaluated  with  and  without 
the  inclusion  of  the  aerospace  knowledge  questions.  In  addition,  the  aerospace 
knowledge  questions  were  deleted  from  the  pilot  BI  key,  leaving  only  the  bio¬ 
graphical  portion  of  the  test  to  be  used  as  a  predictor  variable  in  order  to  make 
direct  comparisons  between  the  old  pilot  BI  key  and  the  new  NFO  BI  keys .  The 
scoring  prpcedures  are  denoted  as  follows: 


1.  Old  BI. 


Present  pilot  key  with  aerospace  knowledge 
questions. 


2 .  New  BI , 


New  NFO  BI  key  based  on  a  dichotomous  criterion , 


3 .  Residual  BI , 


New  NFO  BI  key  based  on  the  residual  criterion. 


4.  New  BI  +  AK.  New  NFO  BI  key  based  on  a  dichotomous  criterion 

with  aerospace  knowledge  questions, 


5 .  Old  BI  -  AK  , 


Present  pilot  key  without  the  aerospace  knowledge 
questions. 


6 .  Residual  BI+AK  New  NFO  BI  key  based  on  the  residual  criterion 

with  aerospace  knowledge  questions. 


In  the  evaluation  of  the  new  BI  keys ,  three  dichotomous  training  criteria 
were  utilized.  The  training  criteria  are  identified  as  follows: 


1.  Pass/DOA.  DOA  (Drop  on  Arrival)  is  a  type  of  voluntary  with¬ 
drawal  which  occurs  during  the  first  twelve  weeks  of  school  in 
Naval  Aviation  Schools  Command . 


Pass/DOR  and  Flight  or  Academic  Failure.  A  DOR  (Drop  on  Request) 
is  a  voluntary  withdrawal  from  training  occurring  after  the  student 
receives  his  commission. 


Pass/Attrite .  Attrite  includes  all  attrition  groups  mentioned  above. 


A  series  of  multiple  regression  analyses  was  performed,  using  a  forward 
selection  procedure.  The  AQT,  MCT,  and  SAT  selection  test  scores  were  forced 
into  the  equation  initially ,  followed  by  one  of  the  BI  scoring  procedures .  The 
purpose  of  this  effort  was  to  determine  to  what  extent  each  BI  scoring  procedure 
accounted  for  additional  variance  beyond  that  provided  by  the  other  selection 
tests . 
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RESULTS  AND  DISCUSSION 

The  results  of  the  analyses  are  depicted  in  Table  III .  In  Table  III ,  and 
other  succeeding  tables ,  each  of  the  BI  scores  represents  the  fourth  variable 
forced  into  the  regression  analyses  and  should  not  be  interpreted  as  an  addi¬ 
tional  variable  added  to  the  prediction  equation.  This  format  was  followed  so 
that  comparisons  of  the  relative  efficiency  cf  the  BI  scoring  processes  could 
readily  be  made . 
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Table  III 

Summary  of  Initial  Validation  Regression  Analysis  for  an 
NFOC  Population  (1039  subjects) 


1  M  V 
&■; 


$ 


Criterion 

Predictor 

Variablo 

Zero  Order 

r 

Multiple 

R 

R2 

Increase  in 

R 

Pass/Attrite 

(1)  AQT 

.172 

,172 

.029 

.172 

(51%  attrition) 

(2)  MCT 

221 

.234 

.055 

.062 

(3)  SAT 

.166 

241 

.058 

.007 

.  Old  BI 

.156 

.244 

.060 

.003 

(4)  BI 

.  New  BI 

.336 

,356 

.127 

.115 

.  Residual  BI 

.376 

.424 

.180 

.183 

.  New  BI  +  AK 

254 

.289 

.083 

.048 

.  Old  BI  -  AK 

.130 

247 

.061 

.005 

.  Residual  BI+AK 

.321 

.342 

.117 

.101 

Pass/DOA 

(1)  AQT 

.121 

.121 

.015 

.121 

(36%  attrition) 

(2)  MCT 

.152 

.163 

.026 

.042 

(3)  SAT 

.111 

.166 

.027 

.004 

.  Old  BI 

.122 

.172 

.030 

.006 

(4)  BI 

.  New  BI 

.325 

.329 

.108 

.162 

.  Residual  BI 

,389 

.409 

.167 

243 

.  New  BI+AK 

235 

244 

.060 

.078 

.Old  81  -  AK 

.100 

.174 

.030 

.007 

.  Residual  BI+AK 

.312 

.316 

.100 

.150 

Pass/DOR,  Flight  or 

(1!  AQT 

213 

.213 

.045 

213 

Academic  Failure 

(2)  MCT 

281 

.296 

.088 

.083 

(33%  attrition) 

(3)  SAT 

213 

.306 

.094 

.010 

.  Old  BI 

.179 

.307 

.095 

.001 

(4)  BI 

.  New  BI 

.321 

.373 

.139 

.067 

.  Residual  BI 

.333 

.423 

.179 

.117 

.  New  BI  +  AK 

249 

.329 

.109 

.023 

.  Old  BI  •  AK 

.148 

.309 

.095 

.003 

.  Residual  BI  +AK 

.302 

.360 

.130 

.054 

All  zero  order  r  s  end  multiple  Rs  are  significant  at  either  the  .05  or  .01  level  of  confidence. 


The  zero-order  correlations  indicate  the  typically  low  but  significant 
relationship  of  the  AQT ,  MCT,  and  SAT  to  the  three  dichotomous  training  cri¬ 
teria.  The  correlations  are  low  because  they  are  based  on  trainees  who  have 
already  qualified  on  these  screening  tests  for  the  naval  aviation  program.  The 
correlations  would  be  substantially  higher  if  the  predictor  variables  were  applied 
to  an  unscreened  subject  population.  The  comparisons  of  the  BI  scoring  keys 
indicate  that  for  all  criteria,  the  newly  developed  BI  key  based  on  the  residual 
criterion  was  superior  to  all  other  BI  scoring  processes .  For  the  Pass/Attrite 
criterion,  the  step  increase  in  the  multiple  R  was  .183,  increasing  the  amount 
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of  explained  variance  of  the  combined  AQT,  MCT ,  and  SAT  from  5.8  percent  to 
18  percent.  The  BI  developed  on  the  Pass/Attrite  criterion  has  a  step  increase 
of  .115,  which  increased  the  amount  of  explained  variance  to  12,7  percent. 

The  most  consistent  finding  of  the  new  scoring  keys  was  the  drop  in  pre¬ 
dictive  power  when  the  aerospace  knowledge  questions  were  included  in  the  BI 
score.  The  zero-order  correlations  indicate  that  these  questions  must  not  be 
related  to  the  criterion  for  NFQs.  Separate  correlations  of  the  aerospace  know¬ 
ledge  questions  to  the  training  criterion  were  obtained  to  substantiate  this  hypo¬ 
thesis.  The  correlational  data  showed  an  r  of  .066  for  the  pass/attrite  group, 
.050  for  the  pass/DOA  group,  and  .076  for  the  pass/DOR  and  flight  or  academic 
failure  group.  These  correlations  are  unusually  low,  indicating  that  the  aero¬ 
space  knowledge  questions  are  of  no  practical  value  for  the  NFO  population. 

For  a  pilot  population,  the  correlation  between  the  aerospace  knowledge  ques¬ 
tions  and  the  Pass/Attrite  training  criterion  showed  that  these  questions  were 
very  useful  in  the  total  BI  score  for  that  group .  The  zero-order  correlation 
was  .164,  which  is  a  substantial  relationship. 

As  previously  indicated  (see  Table  II)  ,  a  major  problem  with  the  use  of 
the  pilot  BI  key  for  the  NFO  population  was  the  high  relationship  of  the  BI  scores 
to  the  MCT  scores .  Table  IV  illustrates  that  the  newly  developed  residual  BI 
key  has  a  lower  relationship  to  the  AQT,  MCT,  and  SAT  than  either  the  new 
BI  based  on  the  dichotomous  criterion  or  the  old  BI .  Additionally  ,  the  new  BI 
key  developed  with  a  true  dichotomous  criterion  does  not  have  as  high  a  cor¬ 
relation  to  the  selection  tests  as  the  old  pilot  BI, 

Table  IV 

Correlation  of  BI  Keys  to  Selection  Tests 


KEYS 

AQT 

MCT 

SAT 

New  BI 

.260 

.414 

.235 

Residual  BI 

.136 

.135 

.117 

Old  BI 

.296 

.519 

.327 

In  order  to  ascertain  if  the  residual  criterion  was  indeed  independent  of 
the  selection  test,  the  relationship  of  the  residual  criterion  score  to  each  selec¬ 
tion  test  was  acquired.  The  results  indicated  that  the  residual  criterion  score 
had  a  .006  relationship  with  the  MCT,  .012  with  the  AQT,  and  .004  with  the 
SAT.  Although  the  residual  criterion  is  unique,  the  BI  items  themselves  are  not 
completely  independent  of  the  other  tests.  As  indicated  in  Table  IV,  the 
residual-keyed  BI  has  a  correlation  of  J36  to  the  AQT,  .135  to  the  MCT,  and  .117 
to  the  SAT . 
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NFO  BI  KEY  VALIDATION 

Although  the  sampling  procedure  was  used  to  minimize  capitalization  on 
chance  relationships,  the  initial  BI  analysis  is  based  on  the  same  sample  that 
was  used  in  development  of  the  new  BI  keys.  Results  could  be  spurious  and 
should  be  validated  on  a  new  sample.  To  validate  the  new  keys,  records  of  all 
NFOCs  who  entered  training  between  January,  1974,  and  January,  1977,  were 
extracted  from  the  NAMRL  data  bank.  Many  of  these,  students  were  still  in 
training  and  had  incomplete  records.  Of  the  607  complete  records  that  were 
used  for  analysis,  291  completed  training  and  316  (52  percent)  attrited .  All 
aspects  of  the  initial  phase  of  the  study  were  maintained  except  for  the  addition 
of  a  new  dichotomous  training  criterion,  In  this  case,  the  previous  pass/DOR, 
academic  or  flight  failure  group,  was  separated  into  two  groups,  pass/DOR  and 
pass/academic  or  flight  failure.  As  before,  a  series  of  multiple  regression 
analyses  was  performed  with  the  AQT ,  MCT,  and  SAT  forced  into  the  equation 
first,  followed  by  one  of  the  BI  scoring  keys. 

Table  V  shows  the  results  of  the  multiple  regression  analyses.  For  the 
Pass/Attrite  criterion  the  AQT,  MCT,  and  SAT  accounted  for  2.9  percent  of  the 
variance,  The  residual  BI  key  increased  the  explained  variance  to  5.4  per¬ 
cent,  the  new  BI  key  developed  on  the  dichotomous  criterion  to  5.3  percent, 
and  the  old  pilot  key  to  3.5  percent.  When  the  aerospace  knowledge  questions 
were  deleted  from  the  old  pilot  BI  key,  the  amount  of  explained  variance 
increased  from  3 .5  percent  to  4.4  percent.  For  the  newly  developed  BI  keys , 
once  again  the  inclusion  of  the  aerospace  knowledge  questions  decreased  the 
predictive  power  of  the  BI . 

For  the  pass/DOA  criterion  group,  only  the  new  BI  keys  had  any  sub¬ 
stantial  improvement  in  accounting  for  additional  variance  beyond  the  selection 
tests .  The  selection  tests  explained  1  percent  of  the  variance  with  the  old  BI 
or  old  BI-AK,  increasing  the  explained  variance  to  1.5  percent,  the  residual 
BI  to  2.6  percent,  and  the  new  BI  to  2,7  percent.  The  same  trend  is  noted  for 
the  pass/academic  or  flight  failure  group.  The  AQT,  MCT,  and  SAT  selection 
tests  accounted  for  7.8  percent  of  the  variance.  When  the  old  BI-AK  was  added, 
the  explained  variance  increased  to  8,5  percent  while  the  residual  key  accounted 
for  9.6  percent  and  the  new  BI  key  accounted  for  9,3  percent.  For  the  pass/ 
DOR  group ,  the  pilot  BI  key  without  the  aerospace  knowledge  questions 
(Old  BI-AK)  accounted  for  more  variance  beyond  the  selection  tests  than  the 
other  keys.  The  explained  variance  of  the  selection  tests  increased  from  4.3 
percent  to  8.4  percent  for  the  BI-AK,  to  8.2  percent  for  the  residual  BI,  and 
7.5  percent  for  the  new  BI.  The  aerospace  knowledge  questions  decreased 
the  amount  of  explained  variance  in  11  of  12  cases.  It  should  be  noted  that  for 
both  the  validation  sample  and  :he  cross-validation  sample,  the  old  pilot  BI 
key  contributes  little  to  the  multiple  R  relationships. 
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Correlation  of  BI  Keys  to  Selection  Tests 


BI  KEYS 

AQT 

MCT 

SAT 

Residual  BI 

.100 

.100 

.140 

New  BI 

.210 

.310 

.220 

Old  Pilot  BI 

.222 

.430 

.300 

CONCLUSIONS 

The  validation  results  indicate  that  the  new  NFO  BI  keys  which  do  not 
include  the  aerospace  knowledge  questions  are  better  predictors  of  NFOC  attri¬ 
tion  than  the  pilot  BI  key  as  it  is  presently  being  used  ,  When  the  aerospace 
knowledge  questiors  are  extracted  from  the  old  pilot  BI  score,  the  predictive 
power  of  that  key  increases,  but  is  not  so  great  as  either  of  the  two  newly 
developed  BI  keys.  Although  the  increase  of  explained  variance  over  the  old 
pilot  BI  without  the  aerospace  questions  is  small,  the  increase  can  be  con¬ 
sidered  significant.  The  increase  does  indicate  that  the  efficiency  of  the  BI 
has  been  imprc'",ed ,  which  should  increase  the  effectiveness  of  the  selection 
test  battery  as  a  screening  device . 

Little  or  no  difference  was  noted  in  the  validity  data  between  the  BI  key 
based  on  the  residual  criterion  and  the  key  based  on  a  dichotomous  criterion . 
Although  decisions  for  establishing  the  sampling  procedures  for  the  item  analy¬ 
ses  were  arbitrarily  selected,  it  can  be  assumed  that  the  methodology  was 
useful,  since  both  keys  were  successfully  validated.  Thus,  it  appears  that 
reliable  item  weights  can  be  obtained  by  the  sampling  methodology  when  the 
subject/item  ratio  is  considered  inappropriate  for  conventional  analyses . 

More  studies  are  needed  in  order  to  know  the  limitations  of  the  item 
analysis  by  sampling  procedure,  and  when  it  can  be  used  appropriately.  A 
monte-carlo  technique  could  be  used  to  ascertain  what  the  subject/itern  ratio 
needs  to  be,  how  many  samples  need  to  be  pooled,  and  how  many  subjects  are 
required  in  each  sample.  In  addition,  the  parameters  of  the  Dj  distribution 
should  be  assessed  so  that  it  can  be  used  as  an  effective  statistic. 

It  was  expected  that  the  residual  BI  would  be  more  effective  than  the  new 
BI  based  on  the  dichotomous  criterion.  The  residual  BI  key  did  remain  rela¬ 
tively  independent  of  the  other  selection  tests  in  the  first  and  second  sample; 
however,  cross-validated  results  Indicated  no  significant  differences  in  the 
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amount  of  variance  explained  by  the  two  keys.  Since  these  results  were 
unexpected,  further  research  is  required  in  order  to  determine  why  the  residual 
criterion  procedures  were  not  more  efficient. 

RECOMMENDATIONS 

Due  to  the  findings  of  the  present  study,  it  is  recommended  that  the  new 
NFO  BI  key,  excluding  aerospace  knowledge  questions  and  developed  with  the 
conventional  dichotomous  criterion,  be  consiuered  for  use  in  the  Naval  Avia¬ 
tion  Selection  Test  Battery.  If  the  new  BI  key  is  not  adopted  for  NFO  selection, 
then  the  aerospace  knowledge  questions  should  be  deleted  from  the  total  pilot 
BI  key  score  for  the  NFO  candidates  in  order  to  increase  the  effectiveness  of 
the  present  scoring  key. 
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^programs,  Research  personnel  at  the  Naval  Aerospace  Mesdical  Research  Labora¬ 
tory  reasoned  that  a  scoring  key  developed  specifically  for  the  NFO  subject  popula¬ 
tion  might  increase  the  efficiency  of  the  Biographical  Inventory  selection  test . 

The  present  report  represents  an  evaluation  of  a  Biographical  Inventory  specifi¬ 
cally  keyed  for  a  naval  flight  officer  population 

«\ 

Results  indicated  that  a  Biographical  Inventory ,  specifically  keyed  for  a 
naval  flight  officer  population,  with  aerospace  knowledge  questions  excluded, 
is  a  better  predictor  of  naval  flight  officer  candidate  attrition  than  the  Biogra  ¬ 
phical  Inventory  keyed  for  a  pilot  population  with  or  without  aerospace  know¬ 
ledge  questions  excluded . 

It  is  recommended  that  the  newly  developed  NFO-BI  key,  with  aerospace 
knowledge  questions  excluded,  be  considered  for  use  in  the  Naval  and  Marine 
Aviation  Selection  Test  Battery.  If  the  newly  keyed  BI  is  not  adopted  for  NFO 
selection,  then  it  is  suggested  that  scoring  of  aerospace  knowledge  questions 
be  omitted  from  the  presently  used  BI  in  order  to  increase  the  effectiveness  of 
naval  flight  officer  selection . 
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